Machine learning to extract quantitative biomarkers from rf spectrums

ABSTRACT

The present disclosure provides for ultrasound systems and methods to pre-process ultrasound data to distinguish abnormal tissue from normal tissue. An exemplary method can include receiving a set of ultrasound data and partitioning the set into a set of windows. The method can then provide for processing the set of windows to determine a power spectrum for each window. The power spectrum for each window can be processed to determine a normalized power spectrum for each window. This normalized power spectrum can be processed for each window with a machine learning model. The method can then provide for displaying an image where each window of the set of windows is displayed using a unique identifier based on the output of the machine learning model.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/772,304 filed Jun. 12, 2020; U.S. patent application Ser. No.16/772,304 filed Jun. 12, 2020 claims priority to and the benefit ofU.S. Provisional Patent Application No. 62/597,537, filed Dec. 12, 2017,and entitled “Machine Learning to Extract Quantitative Biomarkers fromUltrasound RF Spectrums”, the contents of which are herein incorporatedby reference in their entireties.

FIELD

The present invention is directed to ultrasound for classification andidentification of various physiological aspects of a living organism,including tissue and biomarker identification and classification.

BACKGROUND

The following description includes information that may be useful inunderstanding the present invention. It is not an admission that any ofthe information provided herein is prior art or relevant to thepresently claimed invention, or that any publication specifically orimplicitly referenced is prior art.

Ultrasound imaging is widely used in clinical diagnosis and image-guidedinterventions. However, the field is relatively behind other areas ofquantitative image analysis, including MRI, CT and X-ray analysis. Forinstance, many of the quantitative analysis techniques process the datafrom B-mode, after the ultrasound has already been converted into twodimensional images and much additional RF data has been removed. Inother cases, the ultrasound RF data is analyzed using regression andbest-fit models/analysis that yield specific known features forquantitative analysis. At times, these features can be directlycorrelated to specific tissue properties.

SUMMARY

The present disclosure provides for ultrasound systems and methods fordetecting abnormal and normal tissue. An exemplary ultrasound system,according to a first embodiment of the present disclosure, can include atransducer, a memory, a signal processing unit, and one or moreprocessors. The transducer can be configured to output a set ofultrasound data. The memory can contain machine-readable mediumcomprising machine-executable code having stored thereon instructions.The signal processing unit can include the one or more processors, wherethe one or more processors are coupled to the memory. The one or moreprocessors can be configured to execute the machine executable code tocause the one or more processors to perform a series of steps. Theseries of steps can include receiving a set of ultrasound data andpartitioning the set into a set of windows. The steps can then providefor processing the set of windows to determine a power spectrum for eachwindow. The power spectrum for each window can be processed to determinea normalized power spectrum for each window. This normalized powerspectrum can be processed for each window with a machine learning model.The steps can then provide for displaying an image where each window ofthe set of windows is displayed using a unique identifier based on theoutput of the machine learning model.

In some embodiments of the first disclosure, the machine learning modelcan be a k-means model or a model from a deep learning network. This caninclude (but is not limited to) one or a combination of the algorithmtypes of: a convolutional neural network (CNN), restricted Boltzmannmachine (RBM), long short term memory (LSTM) or a capsule network(CapsNet).

In some examples, the machine learning model can be trained usingultrasound images labeled by a radiologist.

In some examples, the machine learning model can be an unsupervisedmodel.

A second embodiment of the present disclosure can provide for a methodof classifying tissue. The method can include first receiving a set ofultrasound data output from at least one ultrasound transducer. The setof ultrasound data output can represent a tissue of a patient. Themethod can then provide for partitioning the set of ultrasound data intoa set of windows. The method can then provide for processing the set ofwindows with a machine learning model. The method can then provide foroutputting a classification of the tissue.

In some examples, the classification can be a cancer status of thetissue.

In some examples, the method can further include processing the powerspectrum for each window to determine a normalized power spectrum foreach window in the set of windows.

In some examples the power spectrum can be taken using a continuous fastFourier transform (FFT).

In some examples, the power spectrum can be taken using a discrete FFT.

A third embodiment of the present disclosure can provide for anothermethod of classifying tissue. This method can provide for firstreceiving a set of ultrasound data output from at least one ultrasoundtransducer. The data can represent a tissue of a patient. The method canthen provide for partitioning the set of ultrasound data into a set ofwindows. The method can then provide for processing the set of windowswith a machine learning module. The method can then provide foroutputting a classification of the tissue.

In some examples of the third embodiment, the step of processing the setof windows with a machine learning model can first include processingthe set of windows to output a power spectrum for each window in the setof windows.

In some examples of the present disclosure, processing the set ofwindows with a machine learning model can first include processing theset of windows to output a time frequency domain processing technique.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, exemplify the embodiments of the presentinvention and, together with the description, serve to explain andillustrate principles of the invention. The drawings are intended toillustrate major features of the exemplary embodiments in a diagrammaticmanner. The drawings are not intended to depict every feature of actualembodiments nor relative dimensions of the depicted elements, and arenot drawn to scale.

FIG. 1 depicts a perspective view of an overview of an ultrasoundsystem;

FIG. 2 depicts a flowchart showing a prior art method of tissueclassification;

FIG. 3 depicts an example of a flowchart showing a method forclassification of tissue using machine learning analysis of ultrasounddata;

FIG. 4A depicts an example image of a mouse tumor using the systems andmethods disclosed herein to segment the image using k-means clustering;

FIG. 4B depicts the original ultrasound image that was processed forFIG. 4A; and

FIG. 5 depicts a schematic diagram of an ultrasound image analysissystem.

In the drawings, the same reference numbers and any acronyms identifyelements or acts with the same or similar structure or functionality forease of understanding and convenience. To easily identify the discussionof any particular element or act, the most significant digit or digitsin a reference number refer to the Figure number in which that elementis first introduced.

DETAILED DESCRIPTION

Unless defined otherwise, technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. Szycher's Dictionary of MedicalDevices CRC Press, 1995, may provide useful guidance to many of theterms and phrases used herein. One skilled in the art will recognizemany methods and materials similar or equivalent to those describedherein, which could be used in the practice of the present invention.Indeed, the present invention is in no way limited to the methods andmaterials specifically described.

In some embodiments, properties such as dimensions, shapes, relativepositions, and so forth, used to describe and claim certain embodimentsof the invention are to be understood as being modified by the term“about.”

Various examples of the invention will now be described. The followingdescription provides specific details for a thorough understanding andenabling description of these examples. One skilled in the relevant artwill understand, however, that the invention may be practiced withoutmany of these details. Likewise, one skilled in the relevant art willalso understand that the invention can include many other obviousfeatures not described in detail herein. Additionally, some well-knownstructures or functions may not be shown or described in detail below,so as to avoid unnecessarily obscuring the relevant description.

The terminology used below is to be interpreted in its broadestreasonable manner, even though it is being used in conjunction with adetailed description of certain specific examples of the invention.Indeed, certain terms may even be emphasized below; however, anyterminology intended to be interpreted in any restricted manner will beovertly and specifically defined as such in this Detailed Descriptionsection.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular implementations of particularinventions. Certain features that are described in this specification inthe context of separate implementations can also be implemented incombination in a single implementation. Conversely, various featuresthat are described in the context of a single implementation can also beimplemented in multiple implementations separately or in any suitablesubcombination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations may be depicted in the drawings in aparticular order, this should not be understood as requiring that suchoperations be performed in the particular order shown or in sequentialorder, or that all illustrated operations be performed, to achievedesirable results. In certain circumstances, multitasking and parallelprocessing may be advantageous. Moreover, the separation of varioussystem components in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

Overview

Conventional ultrasound spectroscopy uses ultrasound frequency spectrumsfrom ultrasound RF data (raw sound signal as it returns into transducer)to extract specific, pre-defined quantitative parameters that candescribe tissue state (e.g., information on the ultrasound scatter size,concentration and change in impedance in tissue).

In some conventional applications, systems extract sets of parameters tocharacterize and classify tissue from RF, or a frequency-domain RF(employing a Fourier or wavelet transform). This process takes the RFdata and first normalizes it to a calibration spectrum (acquiredseparately) to remove the machine setting, some artifacts and some userdependencies. Next, it extracts parameters, which are correlated todifferent tissue pathologies. This is done by taking a certain segmentof the spectrum (with an established bandwidth around the centerfrequency of the signal) and fitting it to a specific mathematical modelor simple linear regression. However, the parameters and quality of fitare not ideal, and predetermined models and parameters usually missvaluable information degrading the accuracy and specificity ofclassification using these methods.

Ultrasound RF data, both beam-formed and pre-beam-forming, contains moreinformation than standard, B-mode or 2-D, ultrasound images as collectedaccording to the conventional methods described above. Though useful,B-mode images, which are perceptible and interpretable by human vision,require discarding the majority of ultrasound RF data to create them.The present disclosure provides for extracting, processing, andcategorizing this normally-discarded information from the ultrasound RFdata. This normally-discarded information can be analyzed using computeralgorithms, such as ML or deep learning methods, to recover and revealsignal information that is beyond human perception. This is unexpectedand surprising, since state-of-the-art applications of artificialintelligence (AI) only enhance recognition and awareness (for example,of photographs or language) within the bounds of human senses,cognizance and comprehension.

The present disclosure therefore uses machine learning to takeultrasound images and to train machine learning models to recognizecertain pathologies based on these conventionally qualitative images.Effectively, this system is used to automate the clinical process of aclinician analyzing images from the processed ultrasound data. However,conventional clinical images (e.g. two dimensional B-mode images) areheavily processed from the raw ultrasound data and though they appeargood qualitatively, they lack information about tissue microstructure.

However, it has been discovered that using artificialintelligence/machine learning methods (“ML”) to analyze the ultrasounddata as is—instead of conventional mathematical models or “fits” thatparameterize the signal—provides for vastly increased accuracy andspecificity. Particularly, this allows for more accurate and specificclassification and identification of tissue types and biomarkers relatedto tissue—including for disease states. Additionally, it has beendiscovered that applying the ML models to the raw data or the frequencyspectrum data—rather than the images (e.g. B-mode) afterprocessing—provides much more accurate results because the alreadyheavily processed images have much of the important ultrasound datafiltered out, including that which is useful for diagnosis andclassification (e.g., a lot of the microstructure of tissue is lost whenprocessing the data to form images).

In some examples, the entire spectrum of data or a specific subset ofthe spectrum can be used as a feature set to train an ML model toevaluate and classify tissues in ultrasound images for screening,diagnostics and treatment monitoring applications. The spectrum-basedfeatures as is, or normalized (using standard methods) can be used asis, or combined with conventional parameters from linear regression ormathematical models. The RF ultrasound data may be converted to anormalized power spectrum which may then be fed into a trained(supervised, unsupervised or deep learning method) ML model(s) toclassify tissues for specific applications, or to find specificmulti-parametric biomarkers, or to identify and apply previously unknownparameters of normal and pathological physiology. In other examples, theraw data before conversion to spectra, or the RF converted to otherdomains (i.e. Laplace transform, principal component analysis, etc.) maybe input to ML learning algorithms.

Ultrasound System

Various conventional ultrasound systems may be utilized that include anultrasound transducer and signal processing equipment. For instance,FIG. 1 illustrates an example of an ultrasound system that includes anultrasound transducer 100, a signal processor 110, a mobile device 105,a network 160, display 150, server 170, and database 180.

In this example, a handheld ultrasound transducer 100 contained in ahandheld case (e.g. ultrasound probe) is utilized. In other examples, arobotic arm transducer or other transducer 100 implementations may beutilized. In some examples, to focus the ultrasound beam, the transducer100 may use phased array techniques, shaping of the transducer 100 orutilize a physical lens.

In some examples, materials on the face of the transducer 100 enable thesound to be transmitted efficiently to the patient's body 105 (e.g.human, animal) by impedance matching. For instance, this may be a rubberor other coating. In addition, a water based gel may be utilized betweenthe patient's skin and the ultrasound transducer case. The sound wavespropagate through the patient 105 until they are reflected in placeswhere the acoustic impedance changes in the patient's 105 body. Thereturn of the sound wave is then detected by the transducer(s) 100,which is converted to a signal that is the RF data. The RF data can bepre-beamforming in or post-beamforming.

The ultrasound data output from the transducer 100 may be processed by aprocessing unit 110 or other associated processors and computing devicesin various ways as disclosed herein. In some cases, after processing animage showing various identified tissues or other indicators may bedisplayed on a display 150. The ultrasound data may be processed fullyor partially by electronics included within the transducer 100 casing,on a separate processing unit 110, or an electrically connectedcomputing device. In other examples, the data can be fully or partiallyprocessed on a local or remote server 170 connected to the processingunit 110 and/or transducer through a network 160.

Processing and display of the ultrasound data may also take place fullyor partially on an associated mobile device 105, or results of theprocessing may be sent to and displayed on a mobile device 105. Anassociated database 180 may store the ultrasound data, models utilizedfor processing the data, and information about various patients andtesting 105. Additionally, a server 170 or separate servers 170 mayprocess training data to form and/or update the models and store them ona database 180.

The system may connect to a variety of different ultrasound transducers100 and/or signal processing units 110. This data maybe aggregated andanalyzed by a server(s) 170 and may be stored on databases 180. The datafrom different processing units 110 could be utilized to continuallyupdate and train a model for processing the data, based on differenttypes of patients and ultrasound units (e.g., for different types oftransducers 100). Accordingly, the RF signal can be at any stage beforeimage formation pre-beamforming, pre-scan conversion, post beamforming,post-scan conversion, etc.

Ultrasound Data Processing Methods

Previously described ultrasound processing methods are illustrated inthe flowchart of FIG. 2 . For instance, most ultrasound systems includea driver that drives the ultrasound signal 200 sent to the transducer100. Then, the transducer 100 vibrates and emits the ultrasound signals205 that are directed toward a patient's 105 body. The transducer thendetects the reflected signals 210, and outputs the ultrasound data orraw RF data 215. Then, in previously described systems, the rawultrasound data is typically broken up into various windows 220. Then,the power spectral for each window 225 may be determined, and thespectrum normalized 230 to calibrate for certain hardware and parameterssettings (to be independent to system and acquisition parametersproperties). Then, the parameters maybe extracted 235 using line of bestfit or other regression models. The system them would display aparametric map or average of the parameters throughout the image 240.

FIG. 3 illustrates an example of an ultrasound processing methodaccording to the present disclosure. As illustrated, the steps throughoutputting of the raw ultrasound data from the transducer 215 arelargely the same. However, as illustrated, once the raw ultrasound datais output from the transducer 100, the processing steps may follow asimilar or different path, but then are ultimately processed by amachine learning model 300.

In some examples, the same steps of breaking the data into windows 220,performing a power spectrum for each window 225, and normalizing thespectrum 230 may be the same. For instance, many applications use aFourier transform to turn the raw ultrasound data into spectral data forfurther analysis. In other examples, the raw ultrasound data may beinput directly into a machine learning model 300, or optionally mayundergo initial filtering techniques and transformations other thanfrequency processing and then may be fit into a machine learning model300 without first converting the data into the frequency domain.

In some examples, the normalized power spectrums output for each window225 may be used as a feature set 245 input into a machine learning model(step 300). Window size can be adjusted in step 335. The spectrum istypically discretized to be representative of the continuous spectrum toobtain the set of features. In some examples, a set of windows of datacan be processed to output a time frequency domain processing time (step305). In some examples, different processing techniques may utilizenon-normalized or normalized spectrums, full spectrum or windows ofspectrums 315, or in other examples the raw RF data may be input intothe machine learning model or any other form of processed RF data. Insome examples, different Fourier transform discretization 325 may beutilized to develop features.

In these examples, the machine learning model 300 utilized may be amodel trained to find a specific type of tissue. For instance, a server170 may constantly process new data to update and train the model torecognize, for instance, liver cancer based on a signature the modelrecognizes. The model may be different kinds of neural networks, deeplearning neural networks, supervised or unsupervised models, or othersuitable types of machine learning models.

Therefore, the present disclosure does not rely on applying artificialintelligence, machine learning, or deep learning to images made withultrasonography (e.g. B-mode) or rely on performing quantitativeultrasound spectroscopy (QUS). Rather, the present disclosure providesfor pre-processing techniques from ultrasound spectroscopy and thenfeeding the power spectrum signal into a trained AI, ML or deep learningmodel. Feeding the power spectrum signal as such is a new step providedfor by the present disclosure, which can avoid parameterization andnormalize the data. Other feature sets, beyond the normalized powerspectrum (NPS), can be used to train and supplement the NPS being fedinto the models of the present invention. Therefore, the presentdisclosure provides for generating clean data and eliminating severalsources of signal error (including system-specific sources ofvariability) by pre-processing the data. This approach additionallyallows for improved training on high quality data normalized to aspectrum, and has the potential to reduce the data set size needed fortraining. In some embodiments of the present disclosure, training canalso be performed on channel data.

RF data is the foundation to all ultrasound systems, and the methodsdisclosed herein allow calibration of signals originating from differentsystems. The present disclosure demonstrates an application of knowledgetransfer. Systems and methods according to the present disclosure canseparate abnormal tissue from normal tissue. This form of tissuecharacterization enables classification and diagnostics. The systems andmethods of the present disclosure can be applied on low-cost,point-of-care (POC) ultrasound systems, which have limited function andare not usually used for diagnostic tissue characterization. The presentdisclosure thus enables POC devices to perform new diagnosticapplications. With the right labeling information for training, thepresent disclosure thus provides for detecting and yielding tissuepathologies from ultrasound RF signal data that are beyond human abilityto decipher. Conventional, current image formation methods, and appliedalgorithms, by contrast to the methods of the present disclosure, remainconstrained to enhancing what a radiologist can see.

EXAMPLES

The following examples are provided to better illustrate the claimedinvention and are not intended to be interpreted as limiting the scopeof the invention. To the extent that specific materials or steps arementioned, it is merely for purposes of illustration and is not intendedto limit the invention. One skilled in the art may develop equivalentmeans or reactants without the exercise of inventive capacity andwithout departing from the scope of the invention.

Several different modes of ultrasound are known. For instance, in A-mode(amplitude mode), a single transducer scans a line through the body withechoes plotted on screen as a function of depth. B-mode or 2D mode usesa linear array of transducers to simultaneously scan a plane through thebody that case be viewed as a two dimensional image on screen.

FIG. 5 illustrates an example of an ultrasound transducer 100 aimed at apatient's body, that would penetrate into the body and potentiallyidentify an abnormal, diseased, or other types of tissue 585 in theliver for example. For instance, the transducer 100 may contain a lineararray of transducers 100 as illustrated in FIG. 5 . Each of thetransducers 100 would emit or vibrate at a certain frequency (ormultiple frequencies) and sound waves would enter the patient's body.For instance, the ultrasound transducers 100 may emit sound pulses witha Gaussian intensity curve (intensity v frequency) around a centralfrequency within the range of, for example, 1-5 megahertz. For instance,the transducer(s) 100 may emit an acoustic wave that it is centeredaround 2 megahertz, and includes frequencies around 2 megahertz withdecreasing intensity to form a Gaussian curve around 2 megahertz.

When the emitted sound waves encounter various structures of the bodythat have different acoustic properties, portions of the sound waves getreflected back to the transducer 100 at the boundary between thestructures. The greater the difference in the acoustic properties, thegreater is the intensity or energy of the reflected waves. Additionally,structures of the body contribute to attenuation of the signal andscattering.

Then, once the reflected sound waves hit one or more transducer(s) thetransducers will record the exact time of flight when the received waveis registered. Additionally, the intensity of the wave or pulse will bedetermined. Using the time of flight and the linear array of transducers100, the system can reconstruct a two dimensional grid or data segmentsof ultrasound RF data that includes an array of frequencies, intensitiesand other data at each data segments 520. Generally, the array offrequencies can be converted into a power spectrum curve 530 that showsthe returned intensity of the acoustic signal at various of frequencies.In some examples, the system will only record different sized bands offrequencies around the emission frequency or a frequency near theemission frequency. In some examples, the bands may be 4, 5, 6, 7, or 8dB wide.

In the case of a linear array of transducers, the system may usebeamforming or other techniques that utilize concepts of destructive andconstructive interference to reconstruct the frequencies and intensitiesfor each data segments of an ultrasound plane along each RF line. Asdiscussed earlier, in some examples, the signals may be processed into atwo dimensional array of data segments that represent different readingsor data at each data segment, and in turn each represent the reflectedultrasound signal at a certain location in the object (e.g., a portionof the liver if the transducer 100 is aimed at a patient's body).

In some examples, a reference will then be applied to each of the datasegments 520 to normalize a reference tissue to the curve obtained fromthe object or patient. In other examples, the raw data output from thetransducers will be processed by a machine learning model 300 to allowit to find the features. In some examples, a power spectrum 535 will betaken of each data segment 520. In some examples a larger window 510 orsliding window 510 or a bounding box 510 (as in a YOLO, “You Only LookOnce,” strategy) or anchor box 510 may be applied to a matrix of datasegments to average or combine using other statistical techniques toprovide a power spectrum 535 or other data set for each window 510.

Then, a machine learning algorithm 300 may be applied to the data setfor each data segment 520, or each window 510. This may be used toclassify the tissue represented in the window 510 or data segments 520.For instance, the system may classify the tissue in the window 510 asabnormal, cancerous, water, bone, or air. In some examples, the systemmay utilize the input of all of the data segments 520 or windows 510 tothen classify each tissue, or classify larger portions of the tissue,such as a region of interest.

Then, once each data segment 520 or window 510 is classified, the systemmay display an image differentiating the tissues 310 by classification.The image could be displayed as a heatmap or other map similar to FIG.4A, or could have numeric values, diagnosis, or other classificationmapped to each section of tissue. In some examples, the system mayoutput whether or not the tissue contains any cancerous cells afterclassifying each of the data segments 520 or windows 510.

By contrast to FIG. 4A, FIG. 4B shows the original ultrasound image thatwas processed according to a method of the present disclosure to yieldFIG. 4A. The contrast between FIGS. 4A and 4B shows the effectiveness ofthe present disclosure at producing data which can be readily analyzedand classified.

Machine Learning Models

In some examples, the machine learning model 300 will be an unsupervisedalgorithm. In other examples, the machine learning model 300 will besupervised. The, machine learning model 300 may take a variety of forms.For instance, the system may utilize more basic machine learning toolsincluding 1) decision trees (“DT”), (2) Bayesian networks (“BN”), (3)artificial neural network (“ANN”), or (4) support vector machines(“SVM”). In other examples, deep learning algorithms or other moresophisticated machine learning algorithms, e.g., convolutional neuralnetworks (“CNN”), or capsule networks (“CapsNet”) may be used.

DT programs are generally used because of their simplicity and ease ofunderstanding. DT are classification graphs that match input data toquestions asked at each consecutive step in a decision tree. The DTprogram moves down the “branches” of the tree based on the answers tothe questions (e.g., First branch: Is the patient male? yes or no.Branch two: Is the patient having trouble urinating? yes or no, etc.).

Bayesian networks (“BN”) are based on likelihood something is true basedon given independent variables and are modeled based on probabilisticrelationships. BN are based purely on probabilistic relationships thatdetermine the likelihood of one variable based on another or others. Forexample, BN can model the relationships between symptoms and diseases.Particularly, if a patient's symptoms or biomarkers levels are known, aBN can be used to compute the probability that a patient has aparticular disease. Thus, using an efficient BN algorithm, an inferencecan be made based on the input data. They are commonly used by themedical domain to represent reasoning under uncertain conditions for awide range of applications, including disease diagnostics, geneticcounseling, and emergency medical decision support system (MDSS) design.

Artificial neural networks (“ANN”) are computational models inspired byan animal's central nervous system. They map inputs to outputs through anetwork of nodes. However, unlike BN, in ANN the nodes do notnecessarily represent any actual variable. Accordingly, ANN may have ahidden layer of nodes that are not represented by a known variable to anobserver.

ANNs are capable of pattern recognition and have been used for themedical and diagnostics fields. Their computing methods make it easierto understand a complex and unclear process that might go on duringdiagnosis of an illness based on input data a variety of input dataincluding symptoms. While still facing steep limitations, ANN hasdemonstrated to be suitable in a clinical decision support system designand other biomedical applications, such as diagnosis of myocardialinfarction, MDSS for leukemia management, and cancer detection.

Support vector machines (“SVM”) came about from a framework utilizing ofmachine learning statistics and vector spaces (linear algebra conceptthat signifies the number of dimensions in linear space) equipped withsome kind of limit-related structure. In some cases, they may determinea new coordinate system that easily separates inputs into twoclassifications. For example, a SVM could identify a line that separatestwo sets of points originating from different classifications of events.

They have been applied practically and are theoretically well-founded,but can sometimes be difficult to understand. SVMs have been applied toa number of biological domains, such as MDSS for the diagnosis oftuberculosis infection, tumor classification, and biomarker discovery.

However, there is a relatively new type of machine learning algorithmthat is capable of modeling very complex relationships that have a lotof variation that are called deep neural networks. Deep neural networks(DNN) have developed recently to tackle the problems of speechrecognition.

In the IT industry fields, various architectures of DNN have beenproposed to tackle the problems associated with algorithms such as ANNby many researchers during the last few decades. These types of DNN areCNN (Convolutional Neural Network), RBM (Restricted Boltzmann Machine),LSTM (Long Short Term Memory) etc. They are all based on the theory ofANN. They demonstrate a better performance by overcoming theback-propagation error diminishing problem associated with ANN.

In some examples, clustering based on k-means, or K-nearest neighborsapproaches may be a useful machine learning model 300. In otherexamples, a K-means clustering methods may be used as machine learningmodels 300. In other examples, principle component analysis can beutilized to separate the data and classify the tissues.

Machine Learning—Training Data

Machine learning models 300 require training data to identify thefeatures of interest that they are designed to detect. For instance,various methods may be utilized to form the machine learning models 300including applying randomly assigned initial weights for the network andapplying gradient descent using back propagation for deep learningalgorithms. In other examples, a neural network with one or two hiddenlayers can be used without training using this technique.

In some examples, the machine learning models 300 will be trained usinglabeled data, or data that represents certain features, specifictissues, diagnosis, or stage of diagnosis of tissues. In other examples,the data will only be labeled with the outcome and the various relevantdata may be input to train the machine learning algorithm.

For instance, to classify a tissue or window of tissue, various machinelearning models 300 may be utilized that input various data disclosedherein. In some examples, the input data will be labeled by correlatingby having a certified radiologist label the tissues, windows 510 or datasegments 520 that include cancer data. Accordingly, the input to themachine learning algorithm for training data may be cancer boxes ornon-cancerous. In other examples, it may include additional labelsincluding water, tissue, bone, and air.

Computer & Hardware Implementation of Disclosure

It should initially be understood that the disclosure herein may beimplemented with any type of hardware and/or software, and may be apre-programmed general purpose computing device. For example, the systemmay be implemented using a server, a personal computer, a portablecomputer, a thin client, or any suitable device or devices. Thedisclosure and/or components thereof may be a single device at a singlelocation, or multiple devices at a single, or multiple, locations thatare connected together using any appropriate communication protocolsover any communication medium such as electric cable, fiber optic cable,or in a wireless manner.

It should also be noted that the disclosure is illustrated and discussedherein as having a plurality of modules which perform particularfunctions. It should be understood that these modules are merelyschematically illustrated based on their function for clarity purposesonly, and do not necessarily represent specific hardware or software. Inthis regard, these modules may be hardware and/or software implementedto substantially perform the particular functions discussed. Moreover,the modules may be combined together within the disclosure, or dividedinto additional modules based on the particular function desired. Thus,the disclosure should not be construed to limit the present invention,but merely be understood to illustrate one example implementationthereof.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In someimplementations, a server transmits data (e.g., an HTML page) to aclient device (e.g., for purposes of displaying data to and receivinguser input from a user interacting with the client device). Datagenerated at the client device (e.g., a result of the user interaction)can be received from the client device at the server.

Implementations of the subject matter described in this specificationcan be implemented in a computing system that includes a back-endcomponent, e.g., as a data server, or that includes a middlewarecomponent, e.g., an application server, or that includes a front-endcomponent, e.g., a client computer having a graphical user interface ora Web browser through which a user can interact with an implementationof the subject matter described in this specification, or anycombination of one or more such back-end, middleware, or front-endcomponents. The components of the system can be interconnected by anyform or medium of digital data communication, e.g., a communicationnetwork. Examples of communication networks include a local area network(“LAN”) and a wide area network (“WAN”), an inter-network (e.g., theInternet), and peer-to-peer networks (e.g., ad hoc peer-to-peernetworks).

Implementations of the subject matter and the operations described inthis specification can be implemented in digital electronic circuitry,or in computer software, firmware, or hardware, including the structuresdisclosed in this specification and their structural equivalents, or incombinations of one or more of them. Implementations of the subjectmatter described in this specification can be implemented as one or morecomputer programs, i.e., one or more modules of computer programinstructions, encoded on computer storage medium for execution by, or tocontrol the operation of, data processing apparatus. Alternatively or inaddition, the program instructions can be encoded on an artificiallygenerated propagated signal, e.g., a machine-generated electrical,optical, or electromagnetic signal that is generated to encodeinformation for transmission to suitable receiver apparatus forexecution by a data processing apparatus. A computer storage medium canbe, or be included in, a computer-readable storage device, acomputer-readable storage substrate, a random or serial access memoryarray or device, or a combination of one or more of them. Moreover,while a computer storage medium is not a propagated signal, a computerstorage medium can be a source or destination of computer programinstructions encoded in an artificially generated propagated signal. Thecomputer storage medium can also be, or be included in, one or moreseparate physical components or media (e.g., multiple CDs, disks, orother storage devices).

The operations described in this specification can be implemented asoperations performed by a “data processing apparatus” on data stored onone or more computer-readable storage devices or received from othersources.

The term “data processing apparatus” encompasses all kinds of apparatus,devices, and machines for processing data, including by way of example aprogrammable processor, a computer, a system on a chip, or multipleones, or combinations, of the foregoing The apparatus can includespecial purpose logic circuitry, e.g., an FPGA (field programmable gatearray) or an ASIC (application specific integrated circuit). Theapparatus can also include, in addition to hardware, code that createsan execution environment for the computer program in question, e.g.,code that constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, a cross-platform runtimeenvironment, a virtual machine, or a combination of one or more of them.The apparatus and execution environment can realize various differentcomputing model infrastructures, such as web services, distributedcomputing and grid computing infrastructures.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, declarative orprocedural languages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, object, orother unit suitable for use in a computing environment. A computerprogram may, but need not, correspond to a file in a file system. Aprogram can be stored in a portion of a file that holds other programsor data (e.g., one or more scripts stored in a markup languagedocument), in a single file dedicated to the program in question, or inmultiple coordinated files (e.g., files that store one or more modules,sub-programs, or portions of code). A computer program can be deployedto be executed on one computer or on multiple computers that are locatedat one site or distributed across multiple sites and interconnected by acommunication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform actions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application-specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only memory ora random access memory or both. The essential elements of a computer area processor for performing actions in accordance with instructions andone or more memory devices for storing instructions and data. Generally,a computer will also include, or be operatively coupled to receive datafrom or transfer data to, or both, one or more mass storage devices forstoring data, e.g., magnetic, magneto-optical disks, or optical disks.However, a computer need not have such devices. Moreover, a computer canbe embedded in another device, e.g., a mobile telephone, a personaldigital assistant (PDA), a mobile audio or video player, a game console,a Global Positioning System (GPS) receiver, or a portable storage device(e.g., a universal serial bus (USB) flash drive), to name just a few.Devices suitable for storing computer program instructions and datainclude all forms of non-volatile memory, media and memory devices,including by way of example semiconductor memory devices, e.g., EPROM,EEPROM, and flash memory devices; magnetic disks, e.g., internal harddisks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROMdisks. The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

CONCLUSION

The various methods and techniques described above provide a number ofways to carry out the invention. Of course, it is to be understood thatnot necessarily all objectives or advantages described can be achievedin accordance with any particular embodiment described herein. Thus, forexample, those skilled in the art will recognize that the methods can beperformed in a manner that achieves or optimizes one advantage or groupof advantages as taught herein without necessarily achieving otherobjectives or advantages as taught or suggested herein. A variety ofalternatives are mentioned herein. It is to be understood that someembodiments specifically include one, another, or several features,while others specifically exclude one, another, or several features,while still others mitigate a particular feature by inclusion of one,another, or several advantageous features.

Furthermore, the skilled artisan will recognize the applicability ofvarious features from different embodiments. Similarly, the variouselements, features and steps discussed above, as well as other knownequivalents for each such element, feature or step, can be employed invarious combinations by one of ordinary skill in this art to performmethods in accordance with the principles described herein. Among thevarious elements, features, and steps some will be specifically includedand others specifically excluded in diverse embodiments.

Although the application has been disclosed in the context of certainembodiments and examples, it will be understood by those skilled in theart that the embodiments of the application extend beyond thespecifically disclosed embodiments to other alternative embodimentsand/or uses and modifications and equivalents thereof.

In some embodiments, the terms “a” and “an” and “the” and similarreferences used in the context of describing a particular embodiment ofthe application (especially in the context of certain of the followingclaims) can be construed to cover both the singular and the plural. Therecitation of ranges of values herein is merely intended to serve as ashorthand method of referring individually to each separate valuefalling within the range. Unless otherwise indicated herein, eachindividual value is incorporated into the specification as if it wereindividually recited herein. All methods described herein can beperformed in any suitable order unless otherwise indicated herein orotherwise clearly contradicted by context. The use of any and allexamples, or exemplary language (for example, “such as”) provided withrespect to certain embodiments herein is intended merely to betterilluminate the application and does not pose a limitation on the scopeof the application otherwise claimed. No language in the specificationshould be construed as indicating any non-claimed element essential tothe practice of the application.

Certain embodiments of this application are described herein. Variationson those embodiments will become apparent to those of ordinary skill inthe art upon reading the foregoing description. It is contemplated thatskilled artisans can employ such variations as appropriate, and theapplication can be practiced otherwise than specifically describedherein. Accordingly, many embodiments of this application include allmodifications and equivalents of the subject matter recited in theclaims appended hereto as permitted by applicable law. Moreover, anycombination of the above-described elements in all possible variationsthereof is encompassed by the application unless otherwise indicatedherein or otherwise clearly contradicted by context.

Particular implementations of the subject matter have been described.Other implementations are within the scope of the following claims. Insome cases, the actions recited in the claims can be performed in adifferent order and still achieve desirable results. In addition, theprocesses depicted in the accompanying figures do not necessarilyrequire the particular order shown, or sequential order, to achievedesirable results.

All patents, patent applications, publications of patent applications,and other material, such as articles, books, specifications,publications, documents, things, and/or the like, referenced herein arehereby incorporated herein by this reference in their entirety for allpurposes, excepting any prosecution file history associated with same,any of same that is inconsistent with or in conflict with the presentdocument, or any of same that may have a limiting effect as to thebroadest scope of the claims now or later associated with the presentdocument. By way of example, should there be any inconsistency orconflict between the description, definition, and/or the use of a termassociated with any of the incorporated material and that associatedwith the present document, the description, definition, and/or the useof the term in the present document shall prevail.

In closing, it is to be understood that the embodiments of theapplication disclosed herein are illustrative of the principles of theembodiments of the application. Other modifications that can be employedcan be within the scope of the application. Thus, by way of example, butnot of limitation, alternative configurations of the embodiments of theapplication can be utilized in accordance with the teachings herein.Accordingly, embodiments of the present application are not limited tothat precisely as shown and described.

1. An ultrasound system comprising: a transducer configured to output aset of ultrasound data; a memory containing machine readable mediumcomprising machine executable code having stored thereon instructions; asignal processing unit comprising one or more processors coupled to thememory, the one or more processors configured to execute the machineexecutable code to the cause the one or more processors to: receive theset of ultrasound data and partition the set of ultrasound data into aset of windows; process the set of windows to determine a power spectrumfor each window of the set of windows; process the power spectrum foreach window of the set of windows to determine a normalized powerspectrum for each window of the set of windows; and process thenormalized power spectrum for each window of the set of windows with amachine learning model; and display an image where each window of theset of windows is displayed using a unique identifier based on theoutput of the machine learning model. 2-12. (canceled)